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(54) Image status detecting apparatus and document image correcting apparatus 



(57) Characters read as images at different angles 
and as mirrored images in accordance with scanning di- 
rections are detected by a character recognition proce- 
dure, and are converted into erecting (upright) images. 
Such character images have low character recognition 
rates, therefore, a character image is extracted (102) 



from the scanned image, a rotated angle and an exist- 
ence of mirroring in which the recognition rate becomes 
maximum are detected (104), Then, the same rotation 
and mirroring are applied to the scanned image, thereby 
capable of converting (105) the scanned image into an 
erecting image,, 
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Description 

[0001] The present invention relates to an image sta- 
tus detecting apparatus detecting a status of an image 
(particularly a document image) inputted through an im- 
age input device such as a scanner and relates to a doc- 
ument image correcting apparatus correcting an image 
inputted through an image input device into an erecting 
(upright) image. Particularly, the present invention re- 
lates to a document image correcting apparatus detect- 
ing a direction of a document by detecting a rotated an- 
gle and an existence of mirroring (reflected-inversion) 
of a character among characters inputted in a state that 
these are rotated and mirrored (reflectively inverted), 
[0002] in recent years, an image scanner has been 
used as an input device for a document image. Small 
handy scanners for manual operation, which are carried 
easily, are developed and manufactured in addition to 
fixed-flatbed scanners which are conventional image 
scanners., 

[0003] As to handy scanners, a user can scan in any 
direction, namely, a user can use a handy scanner flex- 
ibly,, However,- there are cases in that a rotated image 
and a mirrored image are read as images due to a read- 
ing (scanning) direction which is a relative direction be- 
tween a sheet and a scanner, therefore, it is necessary 
to correct the image by rotating or rotating after scan- 
ning. For example, this case occurs when a document 
written horizontally is read from the top of the document., 
[0004] Further, there is a possibility in that a whole 
document is mirrored and inputted. In a document writ- 
ten horizontally, a sentence is usually written from left 
to right. When this document is scanned from right to 
left, the inputted characters are mirrored characters. 
Such mirroring occurs when a back and a face of a per- 
meable sheet such as an OHP sheet is reversed in spite 
of a fixed-flatbed scanner. 

[0005] Thus, when a process such as character rec- 
ognition is applied to the inputted document image, the 
document is not inputted in a proper direction and an 
accuracy of the character recognition lowers in a case 
that a character is rotated and mirrored, therefore, it is 
necessary to correct the document image into an erect- 
ing image after inputting document image in order to di- 
rect the document read in this way to a correct direction. 
[0006] There are conventional document image 
erecting method such as Japan Patent Application Laid- 
Open 8-212298 and Japan Patent Application Laid- 
Open 7-28945- 

[0007] Japan Patent Application Laid-Open 8-21 2298 
discloses, as shown in FIG, 31 , a technique in which an 
inputted character is recognized from four directions 
such as angles of 0°, 90°, 180° and 270°, an accuracy 
value of the character recognition is obtained, and the 
direction having the highest accuracy value is judged as 
the document direction, 

[0008] However, Japan Patent Application Laid-Open 
8-21 2298 gives no description that a mirrored character 



is corrected. For example, though a symmetric charac- 
ter is read conversely, there is a high possibility in that 
the apparatus recognizes this character as normal Fur- 
ther, the accuracy value is obtained from the character 
s recognition result regardless of the reading direction, 
therefore, there is a case that the accuracy value be- 
comes higher than a character read in a correct direc- 
tion. 

[0009] Japan Patent Application Laid-Open 7-28945 
10 discloses atechnique, in which a character is unsuitable 
to Judge the direction of document, such as a left-right 
symmetric character, a upside-down symmetric charac- 
ter, a point symmetric character, a character which looks 
like another character when turned upside down, a char- 
ts acter which looks like another character when mirrored, 
therefore, when this character is clipped, the clipping 
process is re-executed., 

[0010] It is desirable to provide a document image 
correcting apparatus detecting a status of a document 

20 image inputted through an input device such as a scan- 
ner with an image status detecting apparatus, and au- 
tomatically correcting document images into a proper 
image even if a rotated document image, a reversed 
document image or a rotated/mirrored image. 

25 [0011] It is also desirable to determine a character to 
be extracted based on a specific condition when a char- 
acter is extracted in order to use only specific kinds of 
characters for judging the rotation and the mirroring of 
the document. Concretely, characters used for judg- 

30 ment are selected by using a value of which a standard 
value varies in accordance with a kind of a character, 
such as a length-breath ratio, a black (white) pixel ratio 
and a pixel size„ 

[0012] An embodiment of the first aspect of the 

35 present invention can provide an image status detecting 
apparatus for detecting whether or not a character im- 
age inputted through an image input unit is mirrored hor- 
izontally or vertically and/or what degree the character 
image is rotated compared with an original character 

40 The image status detecting apparatus comprises an im- 
age converting unit for rotating and for mirroring the 
character image inputted through the image input unit; 
a character recognizing unit for referring a character rec- 
ognition dictionary storing character feature information 

45 and for obtaining candidate characters resemble to the 
character image mirrored and rotated by the image con- 
verting unit and a recognition rate indicating a degree of 
recognition certainty; and a conversion procedure de- 
tecting unit for detecting, based on the candidate char- 

so acters and the recognition rate obtained by the charac- 
ter recognizing unit, a rotation angle and an existence 
of mirroring needed to direct the character image to a 
proper direction. 

[0013] An embodiment of the second aspect of the 
55 present invention can provide a document image cor- 
recting apparatus using this image status detecting ap- 
paratus The document image correcting apparatus 
comprises a character image storing unit for storing a 
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character image inputted through image input unit; an 
image status detecting unit for detecting whether or not 
the character image stored in the character image stor- 
ing unit is mirrored and what degree the character image 
is rotated compared with an original character; and an 
image correcting unit for directing the character image 
stored in the character image storing means to the prop- 
er direction based on a detected result of the image sta- 
tus detecting means. 

[0014] Further, the image status detecting units may 
comprise an image converting unit for rotating and for 
mirroring the character image inputted through the im- 
age input unit; a character recognizing unit for referring 
a character recognition dictionary storing character fea- 
ture information and for obtaining candidate characters 
resemble to the character image mirrored and rotated 
by the image converting unit and a recognition rate in- 
dicating a degree of recognition certainty; and a conver- 
sion procedure detecting unit for detecting, based on the 
candidate characters and the recognition rate obtained 
by the character recognizing means, a rotation angle 
and an existence of mirroring needed to direct the orig- 
inal character image to a proper direction, 
[001 5] In these aspects, a scanner, more particularly, 
a hand-held scanner is generally used as the image in- 
put unit. A document image including a line (and lines) 
of plural characters is read by the scanner. Then, it is 
preferable that the document image is temporarily 
stored in the document image storing unit (such as a 
memory) and then at one or more specific character is 
extracted from the document image and is stored in the 
character image storing unit (such as a memory). 
[0016] When a hand-held scanner is used, the read 
image is mirrored in a case that the scanning direction 
differs from the original document direction Thus, the 
image status detecting unit detects whether or not the 
character image stored in the character image storing 
unit is mirrored as to the original character When the 
image status detecting unit judges that the character im- 
age is mirrored, the image correcting unit makes the 
character image stored in the character storing unit 
erecting. 

[0017] The image status detecting unit also detects 
what degree the character image is rotated compared 
with an original character, The image correcting unit di- 
rects the character image stored in the character storing 
unit to a proper direction in accordance with the detected 
result of the image status detecting unit. Moreover, the 
image can be corrected by combining the mirroring and 
rotation. 

[0018] In these aspects, document image correcting 
apparatus may comprise a document image storing unit 
for storing a document image including a character line 
which consists of plural characters inputted through the 
image input unit; and a character image extracting unit 
for extracting at least one character image from the doc- 
ument image inputted through the input image unit., Sn 
this case, the character image storing unit stores the 



character extracted by the character image extracting 
unit, and the image correcting unit corrects the docu- 
ment image stored in the document image storing unit 
to a proper image in a specified direction in accordance 

5 with a detected result of the image status detecting unit 
[0019] Further, the document image correcting appa- 
ratus may comprise a conversion controlling unit for 
controlling the image converting unit. That is, when the 
detected result is different from a predetermined result, 

10 conversion by the image converting unit is varied, and 
then the character recognition is tried again based on 
the varied conversion, whereby capable of recognizing 
a character speedily, 

[0020] The conversion procedure detecting unit may 

15 detect a rotated angle and/or an existence of mirroring 
in a case that the recognition rate of the character image 
obtained by the character recognizing unit becomes 
maximum among all combinations of rotation and mir- 
roring executed by the image converting unit. 

20 [0021] The conversion procedure detecting unit may 
detect a rotated angle and an existence of mirroring 
when the recognition rate obtained by the character rec- 
ognizing means is over a predetermined value, 
[0022] The conversion controlling unit may control the 

25 image converting unit so that the character image is con- 
verted in a state that rotation and mirroring are com- 
bined in a specified order For example, priority is given 
to adirection in which scanning is executed frequently, 
[0023] The conversion controlling unit may control the 

so image converting unit so that the character image is con- 
verted in accordance with only a specified combination 
of rotation and mirroring among plural combinations of 
rotated angles and mirroring. A case is assumed in that 
scanning is executed in only specific directions 

35 [0024] The conversion controlling unit may.control the 
image converting unit so that the character image is con- 
verted in order of high frequency among combinations 
of rotated angles and mirroring which previously direct 
character images to the proper direction,. This function 

40 is used to learn a habit of a personal for scanning (a 
person's habits when scanning). 
[0025] The conversion controlling unit may control the 
image converting unit so that a character image is rotat- 
ed every predetermined angle (such as 0°, 90°, 180° 

45 and 270° while mirrored. 

[0026] The image correcting unit does not correct 
character images not satisfying a specified condition as 
a result of character recognition as to all character im- 
ages converted by said image converting means 

50 [0027] An embodiment of another aspect of the 
present invention can provide an image status detecting 
method comprising: an image converting step of rotating 
and/or of mirroring the character image inputted through 
the image input tool; a character recognizing step of re- 

55 ferring a character recognition dictionary storing char- 
acter feature information and of obtaining candidate 
characters resemble to the character image mirrored 
and rotated in the image converting step and a recogni- 
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tion rate indicating resemblance; and a conversion pro- 
cedure detecting step of detecting, based on the candi- 
date characters and the recognition rate a recognized 
result in the character recognizing step, a rotation angle 
and an existence of mirroring needed to direct the orig- 
inal character image to a proper direction, 
[0028] An embodiment of another aspect of the 
present invention can provide a document image cor- 
recting method comprising; a character image storing 
step of storing a character image inputted through an 
image input tool; an image status detecting step of de- 
tecting whether or not the character image stored in the 
character image storing step is mirrored and what de- 
gree the character image is rotated compared with an 
original character; and an image correcting step of mak- 
ing the character image stored in the character image 
storing step erecting based on a detected result of the 
image status detecting step. 

[0029] An embodiment of another aspect of the 
present invention can provide a program (by itself, or in 
the form of a downloadable signal from e.g., the Internet, 
or in recorded form on a carrier which may or may not 
be directly computer-readable). Other references herein 
to programs are to be interpreted likewise. The program 
comprises an image converting step of rotating and of 
mirroring the character image inputted through the im- 
age input tool, a character recognizing step of referring 
a character recognition dictionary storing character fea- 
ture information and of obtaining candidate characters 
resemble to the character image mirrored and rotated 
in the image converting step and a recognition rate in- 
dicating resemblance; and a conversion procedure de- 
tecting step of detecting, based on the candidate char- 
acters and the recognition rate a recognized result in the 
character recognizing step, a rotation angle and an ex- 
istence of mirroring needed to direct the original char- 
acter image to a proper direction. 
[0030] An embodiment of another aspect of the 
present invention may be a computer readable medium 
storing a program. The program comprises a character 
image storing step of storing a character image inputted 
through an image input tool; an image status detecting 
step of detecting whether or not the character image 
stored in the character image storing step is mirrored 
and what degree the character image is rotated com- 
pared with an original character; and an image correct- 
ing step of making the character image stored in the 
character image storing step erecting based on a de- 
tected result of the image status detecting step. 
[0031] An embodiment of the third aspect of the 
present invention is a document image correcting appa- 
ratus comprising a character image extracting unit for 
extracting a character image satisfying a predetermined 
condition from an inputted document image, and a char- 
acter direction detecting unit for detecting a rotation and 
an existence of mirroring as to the character image ex- 
tracted by the character image extracting unit, 
[0032] The predetermined condition may be a ratio of 



a number of black pixels to a number of pixels, a length 
ratio of a long side to a short side, or at least one of a 
number of pixels in a lengthwise direction and a number 
of pixels in a horizontal direction in the character image 
s extracted by the character image extracting unit, 

[0033] The character image extracting unit, when a 
character image having a high symmetry appears, does 
not clip the character image. 

[0034] The character direction detecting means may 
10 apply a character recognition to the character image ex- 
tracted by the character image extracting unit, and when 
a candidate character obtained as a result of the char- 
acter recognition is coincident with a predetermined 
character, does not detect a direction of the character 
is image. 

[0035] An embodiment of the fourth aspect of the 
present invention is a document image correcting appa- 
ratus comprising a character image extracting unit for 
selectively extracting a character image satisfying a pre- 
20 determined condition from an inputted document image, 
a character image converting unit for applying a rotation 
process at a predetermined angle to the character im- 
age extracted by the character image extracting unit, for 
applying the rotation process to the character image af- 
25 ter applying a mirroring process, and for outputting the 
character image to which the rotation process and the 
mirroring process are applied; a character recognizing 
means for executing a character recognition as to each 
direction of the character image outputtedf rom the char- 
ge acter image converting unit; and a document direction 
determining unit for determining a direction of the input- 
ted document image based on a result of the character 
recognition executed by the character recognizing unit 
[0036] In this embodiment, the character recognizing 
35 unrt outputs reliability information indicating an accuracy 
of the character recognition process as to each direction 
of the character image outputted from the character im- 
age converting unit, and the document direction deter- 
mining unit determines the direction of the inputted doc- 
40 ument image based on the reliability information of the 
character image outputted from said character recog- 
nizing unit 

[0037] The document direction determining unit may 
detect a maximum reliability information among the re- 
45 liability information inputted from said character recog- 
nizing means and determines the direction of the input- 
ted document image based on the maximum reliability 
information* 

[003S] The document direction determining unit may 
so compare the reliability information inputted from the 
character recognizing unit with a predetermined thresh- 
old, and executes no document direction determination 
based on the character image when no reliability infor- 
mation exceeds the predetermined threshold as to each 
55 direction of the character image. The document direc- 
tion determining unit may compare the reliability infor- 
mation inputted from the character recognizing unit with 
a predetermined threshold, and executes no document 
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direction determination based on the character image 
when a number of reliability information exceeds a pre- 
determined number 

[0039] The document direction determining unit may 
detect directions of characters as to plural character im- 
ages and determines the direction of the document im- 
age based on the directions which are detected 
[0040] The document direction determining unit may 
calculate an average of reliability of each character in 
each character direction as to plural character images, 
and determines the direction of the document image 
based on the average. 

[0041] The document direction determining unit may 
determine a character direction obtaining a most 
number of maximum reliability among character direc- 
tions having plural maximum reliability of each character 
image as a document direction, 

[0042] The document direction determining unit, 
when maximum reliability are obtained more than a pre- 
determined number as to a character direction, deter- 
mines the character direction as a document image di- 
rection! 

[0043] An embodiment of another aspect of the 
present invention can provide a document image cor- 
recting method comprising: a step of extracting a char- 
acter image satisfying a predetermined condition from 
an inputted document image; and a step of judging a 
rotation and an existence of mirroring as to the character 
image which is selectively extracted. 
[0044] An embodiment of another aspect of the 
present invention can provide a document image cor- 
recting method comprising: a step of selectively extract- 
ing a character image satisfying a predetermined con- 
dition from an inputted document image; a step of ap- 
plying a rotation process at a predetermined angle to 
the character image which is selectively extracted and 
of applying the rotation process to the character image 
after applying a mirroring process; a step of out putting 
the character image to which the rotation process and 
the mirroring process are applied; a step of executing a 
character recognition as to each direction of the char- 
acter image which is outputted; and a step of determin- 
ing a direction of the inputted document image based 
on a result of the character recognition, 
[0045] An embodiment of another aspect of the 
present invention can provide an information storage 
medium storing a program The program comprises a 
step of extracting a character image satisfying a prede- 
termined condition from an inputted document image, 
and a step of judging a rotation and an existence of mir- 
roring as to the character image which is selectively ex- 
tracted. 

[0046] An embodiment of another aspect of the 
present invention can provide an information storage 
medium storing a program; said program comprises a 
step of selectively extracting a character image satisfy- 
ing a predetermined condition from an inputted docu- 
ment image; a step of applying a rotation process at a 



predetermined angle to the character image which is se- 
lectively extracted and of applying the rotation process 
to the character image after applying a mirroring proc- 
ess: a step of outputting the character image to which 

5 the rotation process and the mirroring process are ap- 
plied, a step of executing a character recognition as to 
each direction of the character image which is outputted; 
and a step of determining a direction of the inputted doc- 
ument image based on a result of the character recog- 

10 nition. 

[0047] An embodiment of another aspect of the 
present invention can provide a status detecting appa- 
ratus for detecting what status a document image input- 
ted through an image input unit is as to a proper direc- 
ts tion., The status detecting apparatus comprises a image 
converting means for producing a rotated image or a 
mirrored image from the document image inputted 
through the image input unit, a character recognizing 
unit for applying a character recognition process to each 
zo character included in the rotated image and the mirrored 
image produced by the image converting unit; and a 
conversion procedure detecting unit for arranging the 
document image inputted through the image input unit 
in the proper direction based on an image direction of 
25 which a character recognition rate is highest among the 
rotated image and the mirrored image as a result of 
character recognition by the character recognizing unit 
[0048] Another document image correcting apparatus 
embodying the invention comprises: an input unit for in- 
30 putting a document image; a selecting unit for selecting 
a character image from the document image inputted 
through the input unit; a judging unit for judging which 
direction the selected character image is inputted as to 
a standard direction; and an arranging unit for arranging 
35 all of the inputted document image into the standard di- 
rection based on the judged direction., 
[0049] Another document image correcting apparatus 
embodying the invention comprises: an input unit for in- 
putting a document image; a judging unit for judging the 
40 inputted document image and an existence of mirroring 
as to a standard direction; and an applying unit for ap- 
plying a process including mirroring to the document im- 
age based on a direction of the document image which 
is judged 

45 [0050] Another document image correcting method 
embodying the invention comprises: a step of inputting 
a character; a step of rotating the inputted character in 
a plurality directions and of mirroring the character ro- 
tated in each direction; a step of executing character 

so recognition as to the rotated and mirrored character; a 
step of selecting a character of which a recognition rate 
of the character recognition is highest; and a step of con- 
verting the inputted character into a proper direction 
based on a rotation direction and an existence of mir- 

55 roring of the selected character 

[0051] The above-described elements may be com- 
bined in any way. 

[0052] Reference will now be made, by way of exam- 
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pie, to the accompanying drawings, in which 

FIG. 1 shows scanning directions by a hand-held 
scanner; 

FIGs. 2(a) through 2(h) show images scanned in re- 
spective scanning directions, 
FIG. 3 illustrates a principle of a procedure used in 
erecting the image, 

FIGs, 4(a) and 4(b) show erecting images in spec- 
ified directions; 

FIG, 5 shows a procedure embodying the present 
invention; 

FIG, 6 is a block diagram showing a structure ot the 
first embodiment, 

FIG., 7 is a block diagram showing a structure of a 
conversion procedure detecting unit; 
FIG. 8 shows relations between a display and erect- 
ing directions; 

FIG 9 shows relations between directions of char- 
acter images and conversion procedures for erect- 
ing images; 

FIG. 10(a) shows an inputted image of an image 
conversion sample 1 and FIG, 10(b) shows an con- 
verted image of the image conversion sample 1; 
FIG. 11(a) shows an inputted image of an image 
conversion sample 2 and FIG, 11 (b) shows an con- 
verted image of the image conversion sample 2; 
FIG. 12(a) shows an inputted image of an image 
conversion sample 3 and FIG 12(b) shows an con- 
verted image of the image conversion sample 3; 
FIG, 13(a) shows an inputted image of an image 
conversion sample 4 and FIG 1 3(b) shows an con- 
verted image of the image conversion sample 4; 
FIG, 14(a) shows an inputted image of an image 
conversion sample 5 and FIG 14(b) shows an con- 
verted image of the image conversion sample 5; 
FIG, 15(a) shows an inputted image of an image 
conversion sample 6 and FIG, 1 5(b) shows an con- 
verted image of the image conversion sample 6; 
FIG, 16(a) shows an inputted image of an image 
conversion sample 7 and FIG, 16(b) shows an con- 
verted image of the image conversion sample 7; 
FIG. 17 is a block diagram showing detail of an im- 
age correcting unit; 

FIG, 1 8 is a block diagram showing a document im- 
age correcting apparatus according to the second 
embodiment of the present invention; 
FIG. 19 is a table showing properties of character 
images of respective kinds; 

FIG, 20 is a view showing a sample of a character 
- - ; -image selecting process; 
FIGs, 21 (a) through 21 (h) are samples of detected 
directions; 

FIGs, 22(a) through 22(h) are samples of detected 
directions, 

FIG,. 23 is a block diagram showing details of the 

document direction detecting unit; 

FIG, 24 is a flowchart showing a process executed 



in the document direction detecting unit; 

FIG 25 is a block diagram showing details of the 

character image selecting unit; 

FIG 26 is a flowchart showing a process executed 
5 in the character image selecting unit, 

FIG, 27 is a block diagram showing details of the 

character direction detecting unit, 

FIG, 28 is a flowchart showing a process executed 

in the character direction detecting unit; 
to FIG 29 is a block diagram showing details of the 

character direction determining unit; 

FIG, 30 is a flowchart showing a process executed 

in the character direction determining unit; 

FIG, 31 shows a conventional sample, 

15 

<Embodiment 1> 

[0053] A hand-held scanner used as an image input 
tool for manual operation scans a sheet in any direction 

20 such as from top to bottom and from right to left as 
shown byG)through®in FIG, 1 , Additionally, characters 
are written on the sheet lengthwise (vertically). 
[0054] In FIG, 1, it is assumed that the directionQ)is 
proper for inputting a character, therefore, when the 

2S sheet is scanned in directions except Q) an image is 
read in a status that the image is rotated and mirrored 
according to the scanning direction, For example, when 
there is a Chinese character "-ft" erecting in the sheet 
and the character is scanned in directions® thro ugh® 

30 in FIG. 1, extracted imaged are respectively shown in 
FIGs, 2G)through 3® 

[0055] Data is read from the head of scanner in a main 
scanning direction, and data is read/arranged similarly 
to the moving direction of the scanner in a sub-scanning 
25 direction, 

[0056] In the present invention, it is notable that char- 
acter recognition rate is low as to rotated characters and 
mirrored character images, therefore, character images 
are extracted, a rotated angle at which the recognition 
40 rate becomes maximum is detected and/or an existence 
of mirroring conversion is detected, and rotation and 
mirroring are applied to an original scanned image to 
obtain an image which is automatically erect in a correct 
direction, 

45 [0057] FIG, 3 illustrates a principle of the present in- 
vention. In an embodiment of the present invention, one 
or more character images (extracted character images) 
are extracted from the document image (scanned im- 
age) scanned with the hand-held scanner, and an rotat- 

so ed angle and an existence of mirroring are detected in 
order to obtain an erecting character image. Based on 
detected results, the same conversion, which is applied 
to the extracted character image to obtain an erecting 
character image, is also applied to the originally- 

55 scanned image, thereby obtaining an erecting image 
(corrected image) from the rotated and mirrored image, 
[0058] Incidentally, various kinds of computer such as 
a PC (personal computer) and a work station may carry 
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out the rotation process and the mirroring process for 
the inputted character, the character recognition proc- 
ess and the image conversion process into a proper di- 
rection. Further, an image scanner may carry out all or 
a part of these processes, 

[0059] Now, as to a fixed display (such as a CRT dis- 
play), the top and bottom of the screen is fixed, there- 
fore, an erecting direction is always fixed However, as 
to a display of a portable information processor, there is 
a case in that images are displayed lengthwise and side- 
ways according to how to use the processor and an ori- 
entation of the processor held by hands. In this case, 
when images taken into the portable information proc- 
essor are made erecting, the images are made erecting 
in a predetermined direction, 

[0060] With this operation, as to a portable informa- 
tion processor which displays an image sideways as 
shown in FIG. 4(a), the image is erecting when the proc- 
essor is arranged sideways, and as to a portable infor- 
mation processor which displays an image lengthwise 
as shown in FIG. 4(b), the image is erecting when the 
processor is arranged lengthwise, therefore, it is possi- 
ble to correct images so as to always erect in accord- 
ance with the watching direction 

[0061] An rotated angle and an existence of mirroring 
of an character image extracted from the document im- 
age are detected by using character recognition as de- 
scribed hereinafter 

[0062] Images obtained by rotating the extracted 
character image at angles of 90°, 180° and 270° are 
produced, and images obtained by mirroring the rotated 
images are produced. For example, in FIG.. 5, the char- 
acter image extracted from the scanned image is rotated 
and mirrored in order to obtain images indicated by CD 
through® The character recognition is respectively car- 
ried out for eight kinds of produced images, an image is 
selected of which the recognition rate is highest or of 
which the recognition reliability is maximum based on 
the recognition results, and an rotated angle and an ex- 
istence of mirroring of the selected image are detected. 
In FIG. 5, the image©has the highest recognition rate, 
and is detected as an image obtained by mirroring the 
original extracted character image, and then rotating the 
mirrored image at angle of 90° to the left. The detected 
rotation and mirroring are applied to the original 
scanned image, thereby capable of correcting the orig- 
inal scanned image to a proper image., 
[0063] While the character recognition is applied to 
extracted images sequentially by combing the rotation 
and the mirroring, an extracted image is judged as rec- 
ognized when the recognition rate is over a predeter- 
mined value, and then remaining conversions are 
skipped 

[0064] In FIG. .5, in a case that the recognition rate is 
set to 0.9 for judging that an image is correctly recog- 
nized, it is detected that the recognition rate is over 0.9 
when the image©is recognized by sequentially recog- 
nizing characters from the imaged) 



[0065] And then, the image©is judged as an erecting 
image and the character recognition process is termi- 
nated, and recognition for images @® are skipped, 
whereby a number of times of conversion / recognition 
5 processes become less than a case in that the characte r 
recognition is carried out for all directions, and then the 
processing time is shortened 

[0066] Now, when a document image is scanned by 
manual operation, it is suspected that there are many 

10 cases in that the document image is scanned in a direc- 
tion such as a direction to read a document A document 
image may be scanned in any directions as shown in 
FIG. 1, however, there is a possibility that scanning in 
the direction©is carried out with higher frequency than 

15 other directions 

[0067] So, the order of priority is previously set based 
on frequencies of scanning directions as the result of 
multiple times of scanning, and the rotation and the mir- 
roring directing the scanned image to a proper direction 

20 are applied to the extracted image which is scanned in 
order of priority, namely, in order of frequency in use. As 
a result, it is possible to detect the rotated angle and an 
existence of mirroring of the extracted character image 
at an early stage, 

2S [0068] As another way, the rotation and the mirroring 
are applied to the extracted image as to only specific 
case in that scanning is frequently carried out, thereby 
reducing processes. 

[0069] Additionally, scanning directions are influ- 
30 enced by a personal habit. Then, a histogram of scan- 
ning directions is accumulated and updated whenever 
scanning is carried out, and a procedure of the rotation 
and the mirroring is determined in accordance with the 
frequency in the histogram, thereby capable of absorb- 
35 tng different habits by diversity of operators and capable 
of optimizing the correction. 

[0070] FIG 6 shows an embodiment according to the 
present invention, for example a PC (personal compu- 
ter) and a work station, provided with an image storing 

40 unit 101 storing a scanned image, a character image 
extracting unit 1 02 extracting a character image from the 
scanned image every character, a character image stor- 
ing unit 103 storing a extracted character image, a de- 
tecting unit 104 detecting a rotation angle and an exist- 

45 ence of mirroring used to make the character image 
erect, and an image correcting unit 105 correcting the 
scanned image to an erecting image based on the spec- 
ified rotation angle and the existence of mirroring, 
[0071] A document image correcting apparatus oper- 

50 ates as follows. An image taken by scanning with the 
hand-held scanner is stored in the image storing unit 
101 as a scanned image after pre-processed such as 
removing noises on the image and correcting inclina- 
tion. 

ss [0072] The character image extracting unit 102 clips 
out at least one image of an area determined as a char- 
acter image from the scanned image stored in the image 
storing unit 101 every character and stores them in the 
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character image storing unit 103. 

[0073] The conversion procedure detecting unit 104 
detects a rotation angle and an existence of mirroring in 
order to correct each extracted image into an erecting 
character image 

[0074] The image correcting unit 105 corrects the 
scanned image stored in the image storing unit 101 to 
an erecting image based on the result detected by the 
conversion procedure detecting unit 104 At this time, 
when a proper direction is instructed from outside in ad- 
dition to the detected result of the conversion procedure 
detecting unit 104, the image correcting unit 105 cor- 
rects the scanned image into an erecting image in an 
instructed direction so as to output it, 
[0075] As above described, the scanned image is au- 
tomatically corrected in an erecting direction, 
[0076] FIG, 7 shows an embodiment of the conver- 
sion procedure detecting unit 104 provided with a rotat- 
ing-mirroring unit 1 06 rotating and mirroring the extract- 
ed character image in accordance with an instruction, a 
character recognizing unit 107 recognizing a rotated- 
mirrored character image and outputting a recognition 
reliability such as a recognition rate, and a conversion 
controlling unit 108 controlling rotation / mirroring based 
on the result of character recognition and detecting a 
rotation angle and an existence of mirroring concerning 
the highest recognition rate. Here, explanations will be 
given of a case in that the character recognizing unit 1 07 
outputs a recognition accuracy as the recognition relia- 
bility. 

[0077] The character image extracted from the 
scanned image is inputted into the rotating-mirroring 
unit 106., The conversion controlling unit 108 controls 
the rotating-mirroring unit 106 so that, as shown in FIG., 
5, images rotating by every angle of 90° and images ob- 
tained by mirroring the rotated images are sequentially 
produced (converted) from an inputted character image., 
The character recognizing unit 107 recognizes the con- 
verted character images and informs the conversion 
controlling unit 108 of recognition accuracies. 
[0078] The conversion controlling unit 108, based on 
the recognition accuracies informed from the character 
recognition unit 1 07, detects a rotation angle and an ex- 
istence of mirroring of the highest recognition accuracy 
among all conversion procedures instructed to the ro- 
tating-mirroring unit 106 and outputs a conversion pro- 
cedure. For example, in FIG- 5, it is detected that the 
recognition accuracy of the image©is detected as high 
among images(j)th rough® and "mirroring + left rotating 
by an angle of 90°" is detected as a conversion proce- 
dure„ 

[0079] Additionally, when the recognition accuracy is 
over a specified threshold value, the conversion control- 
ling unit 108 may terminate the conversion procedure 
detecting process after outputting the rotation angle and 
the existence of mirroring at that time. For example, in 
FIG. 5, when a threshold is set over 0 ,9 and processes 
are sequentially carried out from(j)to@ process is 



finished when the image©is obtained and the process- 
es ©and® are skipped 

[0080] The conversion controlling unit 1 06 may previ- 
ously receive a converting order as an order of priority, 
5 and may control the rotation and the mirroring in accord- 
ance with the order of priority, For example, in FIG, 5, 
when the order of priority is © © © ® © © an 
erecting image©is obtained at a time which three times 
of conversions and character recognition are carried 
10 out, whereby the following conversions can be skipped 
[0081] As another way, in the conversion controlling 
unit 108, detected conversion procedures are accumu- 
lated in a histogram, and the conversion controlling unit 
1 08 controls the rotating-mirroring unit 106 so as to carry 
is out conversion in order of frequency. In this way, the de- 
tected conversion procedure is fed back, whereby it is 
possible to detect a conversion procedure efficiently by 
reflecting a condition such as a habit of an operator who 
scans,, For example, when the sheet is scanned in the 
direction ©in FIG 1, the scanned character image is 
shown in FIG. 5 as "extracted character image". And 
then, the histogram is updated, namely, "mirroring +- left 
rotating at an angle of 90°" is increased. When the scan- 
ning inthedirection©in FIG. 1 is repeated, the frequen- 
cy of "mirroring + left rotating at an angle of 90°" is in- 
creased, whereby the conversion is used in accordance 
with the frequency by giving priority. 
[0082] As above described, the rotation angle and the 
existence of mirroring of the character image extracted 
from the scanned image are detected 
[0083] In this way, the rotation angle and the exist- 
ence of mirroring of the character image in the scanned 
image are detected, and rotation angle and the mirroring 
similar to those detected as to the character image are 
applied to the scanned image, whereby it is possible to 
obtain an erecting image automatically from a scanned 
image though the image is scanned in any direction 
such as from top to bottom or from right to left. 

<Character Recognition Process> 

[0084] The explanations are given of the embodiment 
1 as above described., The present invention may use 
a technique of the character recognition process Then, 
a sample of one character recognition process used in 
the present invention will be explained simply. 
[0085] Hereinafter, descriptions will be given of the 
technique disclosed in "Hiroshi, Kamata et aL Sin- 
gakugiho TECHNICAL REPORT OF IEICE PR MU-25 
(1997-03)" published by Institute of Electronics and 
Communication Engineers of Japan (IECE). 
[0086] According to this technique, candidate charac- 
ters as a recognition result are obtained with a small 
quantity of compressed character features at high 
speed, and then a character is recognized with a large 
quantity of character features accurately. This technique 
includes a character extracting process and a character 
recognizing process. 
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[0087] The character extracting process is a process 
extracting an image of one character from a binary im- 
age which is a text document. This process is a bottom- 
up process integrating pixels to constitute a character 
successively, and is carried out in accordance with the 
foilowing procedure, 

[0088] First, as a basic element of a character, con- 
nected areas of black pixels are extracted. Then, a cir- 
cumscribed rectangle of plural connected areas in which 
circumscribed rectangles are overlapped is obtained, 
and the positional relation of them is analyzed, thereby 
determining a direction of a line and extracting a line to 
extract a position Further, circumscribed rectangles or- 
thogonal to the line are regarded as a group, rectangular 
integration in a direction orthogonal to the line is carried 
out in order to obtain circumscribed rectangles of the 
group, and these circumscribed rectangles are integrat- 
ed in the direction of the line, thereby recognizing them 
as a character and taking a circumscribed rectangle of 
the integrated portion as a character circumscribed rec- 
tangle, 

[0089] This charactercircumscribed rectangle is used 
as a standard, and a width position of a vertical charac- 
ter frame is determined based on the line directional po- 
sition of the character circumscribed rectangular which 
is adjacent in the line. 

[0090] In addition to the above-described main proc- 
ess of the character extracting process, this technique 
also carries out a process to remove elements except 
for characters such as a noise and to separate plural 
characters being contact with each other, 
[0091] Then, the character recognition process ac- 
cepts the image of one character extracted in the char- 
acter extracting process and recognizes the character. 
The character recognition process is carried out in ac- 
cordance with the following procedure. 

(1) Normalization 

[0092] According to this procedure, the image of one 
character extracted from a virtual character frame in the 
character extracting process is changed into a normal- 
ized image 48 pixels (length) X 48 pixels (breadth). 
When the length and breadth of the image of this one 
character exceeds 48 pixels, the image is reduced 
equally, and when the length and breadth of the image 
of this one character is 3/4 of the normalized image, the 
image is magnified so that the maximum side fits the 
virtual character frame. 

(2) Feature Extraction 

[0093] A character feature is extracted from the nor- 
malized character image. This character recognition 
takes a border variation in an area obtained by variably 
dividing the character area so that a number of border 
points is constant, as a variable divided border direc- 
tional feature, 



[0094] In other words, after extracting border points, 
the character circumscribed rectangle is divided into 
partial areas so that a number of border points is con- 
stant, and a directional feature volume is obtained con- 

5 cerning the character border in the lengthwise direction 
in the partial rectangle area Then, the feature quantity 
in the partial rectangle area is shaded off between part 
rectangle areas adjacent to each other concerning the 
short side of the area rectangle, thereby obtaining a final 

10 character feature quantity, 

(3) Feature Compression 

[0095] In this procedure, the character feature ex- 
is tracted in the feature extraction process is compressed 
into a compressed f eatu re of a few dimensions. The fea- 
ture compression process includes two steps, which are 
a process extracting an initial compression feature co- 
ordinate axes in order to obtain a few initial coordinate 
20 axes easy to separate distributions of character f eatu res 
and a process calculating a compression feature by tak~ 
ing an original feature coordinate as the compression 
feature concerning a coordinate axis obtained by or- 
thogonlizing the initial coordinate axis 

25 

(3) Rough Classification 

[0096] In this procedure, a recognition candidate 
character is obtained using the compression feature ob- 
30 tained in the feature compression process. The rough 
classification process uses an accuracy-assured high- 
speed classification and a classification by distance cal- 
culation, 

[0097] According to the accuracy-assured fligh- 
ts speed classification, detailed-classified character fea- 
tures are projected in a rough classification feature 
space, characters possible to be projected are stored in 
a candidate character table, and then candidate char- 
acters are searched. 
40 [0098] However, in this procedure, detailed-classified 
character features are directly projected in the rough 
classification feature space instead of a divided area, 
and a margin is added to the projected distribution in 
order to magnify it with the character feature for charac- 
45 ter pattern detailed classification for producing a recog- 
nition dictionary. 

[0099] The classification by the distance calculation 
is used only in a case that a number of candidate char- 
acters by the recognition result is not less than 70. 

so 

(5) Detailed Identification 

[0100] In this procedure, concerning recognized can- 
didate characters outputted in the rough classification 
55 process, distances are calculated between an inputted 
character and an original character feature in the recog- 
nition dictionary and are sorted, and then a candidate 
character of the minimum distance is taken as the rec- 
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ognized result, 

<Sample of Hand-held Scanneo 

[01 01] The hand-held scanner as an image input tool, 
not shown, is provided with an image reading unit opti- 
cally reading an image on a medium to be read such as 
a sheet of paper and converting the image into an ana- 
log signal, a moving quantity measuring unit measuring 
a moving quantity of the image reading unit on the me- 
dium to be read, an image signal processing unit 
processing the analog signal outputted from the image 
reading unit, a read controlling unit driving and control- 
ling the image reading unit in accordance with the mov- 
ing quantity of the moving quantity measuring unit and 
an interface unit outputting read data to an external in- 
formation processor. 

[0102] The image reading unit reads an image in con- 
tact with the mediunatOj be read by using a line sensor, 
a light source (LED or cold cathode tube) and an optica! 
lens. The light source is lighted during reading, and the 
line sensor starts reading with a starting signal, thereby 1 
converting reflected light quantity at each position of the 
sensor and outputting it as an analog signal 
[0103] The moving quantity measuring unit converts 
the moving quantity on the medium to be read into a 
pulse signal with an encoder and outputs it The image 
signal processing unit digitizes the picture signal from 
the image reading unit with an A/D converting circuit, 
and also carries out image processing such as binars- 
zation when needed., 

[0104] The read controlling unit controls ail of reading 
and transfers data to the information processor Stating 
signals are always produced while reading, and line da- 
ta to which image processing is already applied is stored 
in an inside buffer. When the moving pulse from the 
moving quantity measuring unit is detected, the image- 
processed line data stored in the buffer of a line to be a 
subject is transferred to the information processor with 
interruption operation. When two or more moving pulses 
are detected in one driving cycle of the line sensor, the 
scanner moves over the reading capacity of the scan- 
ner, therefore, only one data is transmitted. When no 
moving pulse is detected in one driving cycle of the line 
sensor, the scanner dose not move, therefore, the im- 
age-processed line data stored in the buffer is discard- 
ed, and the next line data is stored. 
[0105] The interface unit acts as an electric interface 
to the information processor, and executes interruption 
and data access., The data transmitted to the information 
processor is displayed by the display unit of the infor- 
mation processor, therefore, image data in the read data 
can be confirmed. 

[01 06] The data is transferred to the information proc- 
essor every line.. Additionally it is possible to make a 
capacity of the buffer in the read controlling unit large, 
in order to transfer a plurality of line data to the informa- 
tion processor together 



<Another Sample of Process> 

[0107] Next, and explanation will be given of a trans- 
formation in that a document image is converted into an 
s erecting image in correspondence to a rotated angle 
and an existence of mirroring detected from a character 
image. 

[0108] The image correcting unit 1 05 converts a doc- 
ument image into an erecting image based on the direc™ 
10 tional detected result detected by the conversion proce- 
dure detecting unit 104, namely, the rotated angle and 
the existence of mirroring of the character image and an 
instruction of an erecting direction from outside. Here- 
inafter, explanations will be given of a converting proce- 
ss dure into an erecting image and an executing proce- 
dure., 

[0109] First, an explanation will be given of a relation 
between the directional detected result and the erecting 
directional instruction, and the procedure converting in- 
to an erecting image„ 

[011 0] Relations between erecting directions and the 
display displaying an image are defined as respective 
erecting directions 1 through 4 as to the direction of the 
display as shown in FIG. 8„ 

[0111] Based on a rotated angle, an existence of mir- 
roring and the erecting directions 1 through 4 instructed 
from outside, a converting procedure converting a doc- 
ument image into an erecting image is directly deter- 
mined as shown in FIG. 9. 

[0112] Symbols A through G in FIG, 9 respectively 
represent converting procedures, and respective con- 
verting procedure are shown in FIGs 10 through 16. 
Conversion procedures A through G are equivalent to 
the following conversion for an input image„ 

A; Rotation at an angle of 90° 
B: Rotation at an angle of 180° 
C: Rotation at an angle of 270° 
D: Right and left mirroring 

E: Rotation at an angle of 90° + right and left mir- 
roring 

F: Rotation at an angle of 180° + right and left mir- 
roring 

G: Rotation at an angle of 270 degrees + right and 
left mirroring . 

[0113] Incidentally, no conversion is necessary when 
the direction of the input image is equal to the proper 
direction.. 

[0114] Next, an explanation will be given of the con- 
verting procedure from scanned document image into 
an erecting image with a sample of the image correcting 
unit 105 shown in FIG. 17, 

[011 5] The image correcting unit 1 05 is provided with 
a scan address controlling unit 201 , a horizontal scan 
address generating unit 202, a vertical scan address 
generating unit 203, and a pixel order converting unit 
204. 
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[0116] The scan address controlling unit 201 deter- 
mines a converting procedure making the document im- 
age erecting based on the detected result of the con- 
verting procedure detecting unit 104 and the erecting 
directional instruction, and controls the horizontal scan 
address generating unit 202 and the vertical scan ad- 
dress generating unit 203 so as to carry out this conver- 
sioa 

[0117] The horizontal scan address generating unit 
202 generates a scan address to scan the image from 
right to left or from left to right every pixel in accordance 
with the instruction from the scan address controlling 
unit 201, 

[0118] The vertical scan address generating unit 203 
generates a scan address to scan the image from top 
to bottom or from bottom to top every pixel in accord- 
ance with the instruction from the scan address control- 
ling unit 201. 

[0119] The pixel order converting unit 204 scans the 
inputted image every pixel in accordance with the scan 
addresses generated in the horizontal scan address 
generating unit 202 and the vertical scan address gen- 
erating unit 203, and the image of which the pixel order 
is converted is outputted. 

[0120] Next, an action of the image correcting unit 1 05 
is explained with a concrete sample., 
[0121] As an example, an explanation will be given of 
a case in that the converting procedure detecting unit 
1 04 detects "rotated angle = 90° and existing mirroring", 
and instructs "proper direction 3" as the erecting direc- 
tion, 

[0122] In this example, the scan address controlling 
unit 201 selects "converting procedure G" (FIG. 16) 
based on the relation shown in FIG. 9 as a converting 
method. Then, the scan address controlling unit 201 , as 
indicated by arrows at the outside of the inputted image 
shown in FIG. 16(a), instructs the horizontal scan ad- 
dress generating unit 202 to generate a scan address 
from right to left, and instructs the vertical scan address 
generating unit 203 to generate a scan address from 
bottom to top, simultaneously. At this time, the scan ad- 
dress controlling unit 201 , as indicated by arrows at the 
inside of the inputted image in FIG. 16(a), controls timing 
of address generation in order to scan all pixels 
[0123] The horizontal address generating unit 202 
generates an address to scan the image from right to 
left every pixel in accordance with the instruction of the 
scan address controlling unit 201 . 
[0124] Similarly, the vertical scan address generating 
unit 203 also generates an address scanning the image 
from bottom to top every pixel in accordance with the 
instruction of the scan address controlling unit 201 
[0125] The pixel order converting unit 204 reads the 
pixel indicated by the scan addresses generated by the 
horizontal scan address generating unit 202 and the ver- 
tical scan address generating unit 203 from the input im- 
age shown in FIG. 16 (a), and outputs it. 
[01 26] With the above-mentioned action, the input im- 



age shown in FIG., 16(a) is converted into the image 
shown in FIG 16(b) The Relation between the input im- 
age and the converted image is indicated by symbols© 
through® It is possible to convert an document image 

s into an erecting image in the same way as to another 
combination of a detected direction and an erecting di- 
rection. As above described, according to the present 
embodiment, an inputted document image can be con- 
verted into an erecting image based on a rotated angle 

10 and an existence of mirroring detected from a character 
; image, 

[0127] According to the document image correcting 
apparatus of the present embodiment, a character im- 
age is extracted out from a scanned image, at least an 

15 existence of mirroring is detected within a rotated angle 
and mirroring, the scanned image is automatically cor- 
rected to a proper image based on the detected result 
Thus, though an image is scanned with a hand-held 
scanner in any direction such as from top to bottom or 

20 trom right to left, an erecting image can be always ob- 
tained, thereby improving an operational convenience 
and'an user interface. 

{{Embodiment 2» 

25 

[0128] FIG„ 18 is a block diagram showing a main part 
of a document image correcting apparatus according to 
the second embodiment of the present invention. In the 
second embodiment, a character used to judge the ro- 
30 tation / the existence of mirroring in the document image 
* is extracted in accordance with specific conditions, 
whereby specific the document image is corrected ef- 
fectively., 

[0129] As shown in FIG 18, the document image cor- 

35 recting apparatus is provided with a document image 
storing unit 301 temporarily storing document images 
inputted through a not-shown input tool such as a scan- 
ner, a document direction detecting unit 302 detecting 
rotation / mirroring directions of the inputted document, 

40 and an image correcting unit 303 applying an erecting 
process to the inputted document image based on the 
document direction detected by the document direction 
detecting unit 302. With this document image correcting 
apparatus ; the image correcting unit 303 outputs an 

45 erectly corrected document image. 

[0130] As explained in the embodiment 1 , FIGs. 2(a) 
through 2(h) show inputted character images by scan- 
ning. These are character images obtained by scanning 
a Chinese Character "K". These are character images 

so rotated at 0° , 90°, 1 80°, 270° compared with an erecting 
image and respectively mirrored. In this embodiment, 
the character recognition process is applied to the 8 di- 
rections, and the rotation direction and an existence of 
mirroring are judged, 

55 [0131] According to a kind of a character, there are a 
character of which a rotation and an existence of mirror- 
ing are easily judged, and a character image of which a 
rotation and an existence of mirroring are not easily 
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judged. This difference produces an effect on the rec- 
ognition rate of the character recognition For example, 
as to Hiragana Characters and Katakana Characters, 
the character recognition rate becomes low, however, 
as to Chinese Characters, the character recognition rate 
becomes high. Though it is tried to judge the existence 
of rotation / mirroring by using Hiragana Characters or 
Katakana Characters, the character recognition rate is 
low. As a result, an accuracy for direction detection be- 
comes low Then, in this embodiment, a character of 
which the recognition rate is high (such as a Chinese 
character) is selected from the document, and the rota- 
tion / mirroring is detected by using this selected char- 
acter, 

[0132] FIG , 19 shows a black pixel ratio, a long-side 
to short-side ratio and a reliability in a extracted charac- 
ter area for each character such as Chinese Character, 
Hiragana Character, Katakana Character, alphanumer- 
ic, and symbol Each value shown In FIG. 19 shows an 
average for each character Here, the black pixel ratio 
shows a ratio of black pixels to all pixels. The long-side 
to shorter-side ratio shows a value obtained by dividing 
the length of the long side by the length of the shorter 
side in the extracted character area. The reliability 
shows an accuracy of the character recognition, and as 
the value is high as the character recognition rate is 
high. 

[0133] As shown In FIG. 19, it is understood that a 
Chinese character has the highest reliability among five 
kinds of characters. Other characters have relatively low 
reliability compared with a Chinese character., There- 
fore, it is desirable to use Chinese character in order to 
judge the rotation / the existence of mirroring. 
[0134] The value shown in FIG, 19 is an average, 
therefore, as to about a half of Chinese characters, the 
black pixel ratio exceeds 27 %. And, as to about a half 
of Chinese characters, the long-side to short-side ratio 
is 1 . 04 or less„ In other words, when characters satisfy- 
ing a condition that the black pixel ratio is 27 % or more 
and the long-side to short-side ratio is 1 .04 or less are 
selected, a half of Chinese characters can be selected 
[0135] Now, a consideration is given to other charac- 
ters. For example, as to Hiragana characters, the aver- 
age of black pixel ratios is 16 %, and this average is 
lower than the average 27 % of Chinese characters. 
Then, this value is the average, therefore, it is under- 
stood that at least a half of Hiragana characters have 
black pixel ratios 1 6 % or less. On the other hand, as to 
a half of Hiragana characters, black pixel ratios exceed 
1 6 %, however, there is little possibility in that the black 
pixel ratios exceeds 27 % because of the nature of Hira- 
gana character though it is influenced by a kind of font, 
namely, the possibility is substantially zero. That goes 
for Katakana characters, too, Accordingly, it is possible 
to select a half of Chinese characters based on whether 
black pixel ratio exceeds 27 % or not, and it is possible 
to increase Chinese characters as candidates to be se- 
lected compared with Kana characters 



[01 36] Then, as to alphanumeric and symbol, averag- 
es of black pixel ratios are about 28 %. When a consid- 
eration is given to errors while reading, it is expected 
that it is difficult to distinguish between alphanumeric 

s and symbol only based on black pixel ratios. However, 
alphanumeric and various symbols tend to become 
lengthwise compared with Chinese characters of which 
shapes are almost perfect squares. This tendency is 
shown by the long-side to short-side ratio in FIG, 1 9. It 

io is possible to consider that there is little case that the 
long-side to short-side ratio is less than 1 ,04 concerning 
alphanumeric and so on. Then, by adding the long-side 
to short-side ratio to the black pixel ratio, a half of Chi- 
nese characters are selected, however, alphanumeric 

15 or symbol is hardly selected, 

[0137] Further, while extracting characters, there is a 
case in that plural character is misunderstood as one 
character image and extracted, and there is a case in 
that portions (left-side portion, right-side portion and so 

20 on) of a Chinese character are misunderstood as sep- 
arated character images and extracted. In these cases, 
it is expected that the ratio of long-side to short-side is 
different from usual Chinese character. Thus, the long- 
side to short-side ratio is used in order to exclude these 

25 misunderstood characters, 

[01 38] In this way, it is judged whether each character 
is selected or not based on the condition shown in FIG, 
19, whereby it is possible to extract Chinese characters 
efficiently 

30 [0139] FIG, 20 shows each black pixel ratio, long-side 
to short-side ratio and pixel size of each extracted char- 
acter in a part of a sentence and a character selection 
condition. The character selection condition prescribes 
that the black pixel ratio is 25 % or more and 60 % or 

35 iess and the long-side to short-side ratio is 1,10 or less 
in counting on errors for both ratios. Further, as to the 
pixel size, it is prescribed that the extracted area in- 
cludes 30X30 pixels or more., 

[0140] Here, it is assumed that characters in the doc- 
40 ument is written in Mincho typeface, a reading resolution 
is 400 dpi and a character size is set to 6 points or more 
Incidentally, each value shown in FIG, 20 is nothing but 
an example. 

[0141] The character size is set to 30X30 pixels or 
45 more, since the character size influences an accuracy 
of character recognition. Generally, there is a possibility 
in that the accuracy of character recognition lowers 
when a very small character is scanned. In this embod- 
iment, Chinese characters expected to have high accu- 
se racy of character recognition are selected, and it is not 
wise to select a character of which the size is small since 
the recognition rate is low, 

[0142] In a general document, characters between 6 
points and 20 points are used frequently in experience, 
55 Thus, it is very advantageous to detect characters of 6 
points or more. In this embodiment, 30x30 pixels are 
equivalent to 6 points, since the character size is ob- 
tained by "character size = point number X resolution ^ 
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72". When the point number is set to 6 points and the 
resolution is set to 400 dpi, the character size is shown 
by "6x4004-72=33. 333.. ", therefore, the character size 
is considered as roughly 30x30 pixels 
[0143] In addition, no upper limit ot the pixel size is 
particularly set in the case shown in FIG. 4, however, it 
is not possible to deny a possibility in that a part of a 
drawing is extracted as a very big character. In this case, 
since the character recognition is not completed for a 
extracted "character", it may be possible to set an upper 
limit of the character size and not to select a drawing 
when the drawing is extracted as a character As above 
described, it is considered that the size of 20 points is 
considered as an upper limit of the character size used 
in the document, therefore, a number of pixels corre- 
sponding to 20 points may be set as the upper limit of 
the character size. V 
[0144] Further, there is a case that the reading reso- 
lution varies with a scanner, and there is an apparatus 
capable of setting plural resolutions in accordance with 
kinds of scanners. In this case, though characters of the 
same size are read, pixel sizes in the character area 
very vary., In brief, when resolution becomes two times, 
numbers of pixels of the lengthwise direction / sideways 
direction of the character image become 2 times. There- 
fore, it becomes important to change the extracting size 
of the character image in accordance with the resolution 
of the scanner. 

[0145] When a character is read with a scanner, a 
reading resolution is previously set, therefore, it is pos- 
sible to use this reading resolution in order to set a size 
of a character image according tothe reading resolution. 
It is possible to calculate a pixel size based on the char- 
acter size and the resolution, therefore, after setting a 
resolution for reading, an upper limit/ a lower limit of the 
pixel size may be calculated based on the character size 
and the set resolution. Further, a table showing a rela- 
tion between the resolution and the character size is pre- 
viously set in the apparatus, whereby it is possible to 
read a pixel size from the table and to set the pixel size, 
[0146] Now, various symbols or graphics exist in a 
document in addition to general characters., Concerning 
a word-processors and the like, symbols such as black- 
colored square and circle are usually used, and there is 
a case in that a photograph and a drawing exist in the 
document, There is a case in that a character is black- 
compressed under circumstances of printing. As to such 
a character, though the character recognition is carried 
out, it may be impossible or difficult to detect a direction. 
[0147] Concerning these black-colored symbols and 
compressed characters, there are many cases in that 
the black pixel ratio is very high compared with an usual 
character therefore, it is important not to select these 
symbols and characters. Then, in this embodiment, an 
upper limit is also set in addition to the lower limit of the 
black pixel rate. As a result, it is possible to exclude too 
black "character" from selection objects When using 
only the condition such as the long-side to short-side 



ratio, the pixel size and the lower limit of the black pixel 
rate, there is a possibility in that these "characters" sat- 
isfy this condition, therefore, it is very effective to set an 
upper limit of the black pixel ratio, 

5 [0148] Further, there are cases in that the deepness 
of the document image is generally uneven, in that doc- 
ument image is generally light and in that document im- 
age is generally deep. When the range of the black pixel 
ratio is set similarly to that shown in FIG, 20, there is a 

10 possibility in that the following problems occur, 

[0149] When the document image is generally deep, 
it is expected to generally raise the average of the black 
pixel ratio for each kind of characters shown in FIG. 1 9, 
In the case shown in FIG 4, the black pixel ratio is set 

is to 25 % or more, however, the possibility increases in 
that Hiragana character or the like is included in this 
range according to the depth of the document image. 
Accordingly, there is a problem in that Hiragana charac- 
ters not suitable for character direction judgement are 

20 frequently selected. 

[0150] On the other hand, when the document image 
is generally light, it is expected to generally tower the 
average of the black pixel ratio for each kind of charac- 
ters shown in FIG , 19, Then, the possibility lowers in that 

25 Hiragana characters are included in the range of the 
black pixel ratio shown in FIG. 20, however, it is expect- 
ed that a number of Chinese characters in this range 
decreases. Accordingly, in this case, data for character 
direction judgment decreases, 

30 [0151] Then, in order to cope with such a case, the 
range of the black pixel ratio are dynamically varied. 
When the document image is generally deep, for exam- 
ple, the black pixel ratio shown in FIG. 4 is changed from 
25 % to 28 %., On the contrary, when the document im~ 

35 age is generally light, for example, the black pixel ratio 
shown in FIG, 4 is reduced to 22 % from 25 %, 
[0152] Incidentally, the upper limit of the black pixel 
ratio is not shift in this case, since it is considered that 
the upper limit gives little influence to the character se- 

40 lection. 

[0153] Returning to FIG, 20, in this sample, Chinese 
characters "g" and "I" satisfy three above-mentioned 
conditions. Therefore, as characters judging the char- 
acter rotation / mirroring, these characters are selected. 

45 [0154] In this embodiment all of Chinese characters 
are not selected, and there is also a possibility in that 
characters except for Chinese characters are selected. 
The reason is that there are about a half of Chinese 
characters not satisfying the condition such as the black 

50 pixel ratio and the long-side to short-side ratio. In the 
example shown in FIG. 20, the Chinese character "± H 
has low black pixel ratio. It is expected that the character 
recognition rate becomes low concerning such a Chi- 
nese character, therefore, this Chinese character is not 

55 selected in spite of a Chinese character Further, as to 
the Chinese character "ft " at the top of the document, 
the long side is relatively longer than the short side, 
therefore, the possibility remains in that the character is 
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alphanumeric or the like. Therefore, the Chinese char- 
acter "D " is not selected in the case shown in FIG 20. 
[0155] Giving a consideration to the general tenden- 
cy, the rate for selecting Chinese characters having high 
reliability is larger than the rate for selecting other char- 
acters, therefore, it is possible to achieve the object of 
this embodiment, 

[01 56] FIGs 21 (a) through 21 (b) show reliability when 
a Chinese character "I " is read in eight directions in- 
cluding mirrored characters. It is assumed that the char- 
acter is inputted in a state shown in FIG. 21(a) when 
inputted. Then, character recognition is executed for 
character images obtained by rotating the inputted char- 
acter at angles ot 0°, 90<\ 180° and 270° and. for char- 
acter images obtained by mirroring the inputted charac- 
ter and rotating them at angles of 0 D , 90° , 1 80° and 270°, 
and reliabilities are calculated as a result of the charac- 
ter recognition. Reliabilities are obtained by judging 
what degree characters stored in the dictionary and the 
inputted characters are accurate in the character recog- 
nition. Reliability about the eight directions are respec- 
tively shown at left sides of the characters., When these 
eight characters are compared, FIG 21(f) shows the 
highest reliability (992), therefore, it is possible to judge 
that the character shown in FIG , 21(f) indicates a char- 
acter erecting direction . Then, according to this judged 
character direction, the erecting process is executed for 
the whole document 

[0157] FIGs 22(a) through 22(b) shows character im- 
ages of the Chinese character "H" to which the same 
process explained in FIGs, 21(a) through 21(b) is ap- 
plied. The condition and the like for inputting the char- 
acter in FIGs. 21 (a) through 21 (h) are equal to those in 
FIGs 22(a) through 22(h), therefore, it should be origi- 
nally judged that FIG. 22(f) shows an erecting direction. 
[0158] However, the Chinese character "H" is almost 
symmetric, therefore, though this character is mirrored, 
it is impossible to execute the character recognition with 
a character unit.. Accordingly, there is a possibility in that 
the character image shown in FIG, 22(b) is judged as a 
correct direction. In this case, data capable of judging 
whether FIG, 22(b) or FIG., 22(f) is correct is only the 
reliability. The reliability in FIG, 22(b) is larger than that 
in FIG. 22(f) by only "1", therefore, FIG, 22(b) is judged 
as a correct direction though the character is mirrored, 
[0159] In order to exclude the possibility producing 
such a problem, it is judged whether or not the inputted 
character is suitable to direction judgment, for example, 
in a case that the inputted character is symmetry, 
[0160] As a concrete procedure, the following steps 
will be discussed 

[0161] First, the extracted character area is divided 
into two such as top and bottom or right and left at a 
central part. In FIGs, 22(a) through 22(h), the Chinese 
Character "H " is divided in two such as right and left. 
Then, a correlation is obtained for each pixel between 
one divided image and another divided image which is 
mirrored When the correlation between both is very 



close, the character is judged as symmetry in right and 
left (or upper and lower), and this character is not used 
as data for judging the rotation / mirroring of the docu- 
ment, 

$ [01 62] As another procedure, it is possible to set a ta- 
ble storing characters symmetrical at a reference line 
(point) in the horizontal direction or in the vertical direc- 
tion and a table storing characters impossible to be dis- 
tinguished from another character as the result of the 

10 rotation / mirroring. When the rotation / mirroring of a 
character is judged, it is judged whether or not a sym- 
metric character is set in the table When a symmetric 
character is set, the character is not used for rotation 
judgment. 

15 [01 63] When examining the reliability obtained as the 
result of character recognition of the character image in 
each direction, there is a case in that the reliability is 
similarly low in each direction, This Indicates that the 
reliability of the character recognition is low in any direo- 

20 tion, therefore, it is possible to regard that the character 
recognition is not completed 

[0164] It is desirable not to use such a character im- 
age as a reference for the character direction judge- 
ment. Therefore, a threshold of the reliability is previ- 

25 ously set in order to judge that the character recognition 
is carried out correctly. Then, the reliability calculated in 
each direction is compared with this threshold. When 
the reliabilities in all directions are below this threshold, 
the character recognition is not completed, therefore, 

30 the character direction detection is not carried out based 
on this character image. As a result, it is possible to im- 
prove the accuracy of the character direction detection, 
[0165] Further, there is a case in that the high reliabil- 
ities are shown in plural directions as the result of cal- 

55 culating the reliability in each direction. In this case, a 
possibility can be considered in that the character image 
is mirrored as shown in FIG., 22(b) and a possibility can 
be considered in that the character image is recognized 
as a different character, Therefore, a threshold is set to 

40 judge that the character recognition is correctly carried 
out,, When there are two or more directions in which the 
reliabilities exceed this threshold as to one character im- 
age, it is judged that there are some errors in the char- 
acter recognition. Then, the character direction is not 

45 detected about this character 

[0166] Now, concerning the character recognition 
while changing the character direction, there is a case 
in that the character recognition is processed in parallel 
in respective directions, however, generally, thecharac- 

50 ter recognition is sequentially processed. In this case, 
the character recognition is carried out in each character 
direction, and the direction detection process is termi- 
nated when two reliabilities over the threshold are ob- 
tained, thereby unnecessary to carry out the following 

55 process of the character recognition In other words, 
when a reliability exceeds the threshold in the first di- 
rection and a reliability exceeds the threshold in the sec- 
ond direction, the direction detection process is termi- 



27 



EP 0 953 932 A2 



28 



nated, thereby unnecessary to carry out the character 
recognition as to remaining six directions The character 
recognition take comparatively a time, therefore, this 
procedure is very effective to make the process high- 
speed and efficient 

[0167] When the character direction is judged by pay- 
ing an attention to only one character, there is a possi- 
bility in that the document image is corrected in a wrong 
direction by mis-judging the character direction 
[01 68] Therefore, it is desirable to detect character di- 
rections of plural characters and to correct the document 
image based on the detected result, 
[0169] For that purpose, a procedure can be men- 
tioned in that a reliability is calculated for each character 
direction concerning N pieces of characters and an av- 
erage of reliabilities is calculated for each direction, and 
then the direction having the highest average is judged 
as a correct character direction.. With such a counter- 
measures, it is possible to improve an accuracy of the 
character direction detection rather than a case in that 
a direction is judged based on only one character 
[0170] Another method can be mentioned in that each 
character direction is dependently detected as to N piec- 
es of characters and the character direction most fre- 
quently detected among the N pieces of characters is 
judged as a correct document direction. 
[0171] Further, when characters having the same 
character direction continues, that character direction is 
supposed as the correct direction . Therefore, when it is 
detected that the same character direction is continu- 
ously detected as to a predetermined number of char- 
acters, that character direction is judged as a correct 
character direction. 

[0172] There will be other various procedures, and a 
procedure of detecting the document direction with plu- 
ral characters is very effective, 

[0173] When it is impossible to detect the direction 
from the character images, there is a possibility in that 
the document can not be corrected in erecting. In this 
case : a document direction can be previously set to the 
apparatus. When it is impossible to detect the direction 
based on the character image, the document direction 
is determined based on the previously set document di- 
rection and the document can be corrected in erecting 
[0174] It is desirable to set a direction most frequently 
scanned with a scanner as the previously set document 
direction. In this case, though it is impossible to detect 
the document direction based on characters, the possi- 
bility capable of correcting the document image to an 
erecting image becomes very high. 
[0175] FIG. 23 is a block diagram showing the detail 
of the document direction detecting unit shown in FIG , 
1 8. As shown in FIG., 23, the document direction detect- 
ing unit is provided with a character image extracting 
unit 401, a character image selecting unit 402, a char- 
acter direction detecting unit 403 and a document direc- 
tion determining unit 404., 

[0176] FIG 24 is a flowchart showing a process pro- 



cedure executed in the document direction detecting 
unit shown in FIG. 23 Hereinafter, explanations will be 
given of the operation in the document direction detect- 
ing unit with reference to FIGs 23 and 24. 

s [0177] The character image extracting unit 401 clips 
a character image from the inputted document image 
(S001 ) , The character image is extracted in accordance 
with a character image extracting instruction from out- 
side. The character image extracted by the character 

70 image extracting unit 401 is inputted into the character 
image selecting unit 402, Then, it is judged whether or 
not the extracted character image satisfies the previous- 
ly set selection condition (such a condition shown in FIG. 
20)(S002), and the character image judged as satisfying 

is the condition is selected and outputted (S003). 

[0178] The character image selecting unit 402 in- 
structs the character image extracting unit 401 to clip 
another character image when the inputted image does 
not satisfy the set condition. 

20 [0179] The character direction detecting unit 403 de- 
tects the character direction of the character image se- 
lected by the character image selecting unit 402, con- 
cretely, judges the rotated angle and an existence of mir- 
roring, and then detects the character direction based 

25 on the rotated angle and the existence of mirroring 
(S004) 

[0180] The character direction detecting unit 403 also 
counts direction detected results, and instructs the char- 
acter image extracting unit 401 to clip a character image 
30 until obtaining a predetermined number of character di- 
rection results (S005). 

[0181] Then, the character direction determining unit 
404 determines the character direction of the inputted 
character image (S006), 
35 [0182] FIG. 25 is a view showing details of the char- 
acter image selecting unit 402 shown in FIG. 23. FIG, 
26 is a flowchart showing process in the document im- 
age selecting unit 402. 

[0183] Hereinafter, explanations will be given of the 
40 character image selecting unit 402 with reference to 
FIGs. 25 and 26. 

[0184] The character image extracted by the charac- 
ter image extracting unit 401 is respectively inputted into 
the black pixel ratio detecting unit 405, the iong-side to 
45 short-side ratio detecting unit 406, the image size de- 
tecting unit 407, and the character image outputting unit 
409 in parallel. 

[01 85] The black pixel ratio detecting unit 405 detects 
the black pixel ratio of the inputted character image, and 

50 outputs the detected result The long-side to short-side 
ratio detecting unit 406 detects the iong-side to short- 
side ratio of the inputted character image, and outputs 
the detected result. The image size detecting unit 407 
detects the pixel size of the inputted character image, 

55 and outputs the detected result Theses detected results 
of these unit are inputted into the selection judging unit 
408 (St 01) 

[0186] The selection judging unit 408 judges whether 
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or not the detected results inputted from respective 
units, namely, the black pixel ratio, the long-side to 
short-side ratio and the pixel size satisfy a predeter- 
mined condition (S102) When there is one or more de- 
tected results not satisfying the predetermined condi- s 
tion, it is judged this inputted character is unsuitable for 
the direction detection, therefore, this character is not 
selected. Then, the selection judging unit 408 instructs 
the character image extracting unit 401 to clip the next 
character image (S103). io 
[0187] On the other hand, when the character, image 
satisfies the predetermined condition, the selection 
judging unit 408 gives an instruction to the character im- 
age outputting unit 409 (S104), The character image 
outputting unit outputs the inputted selection character ?s 
image based on the instruction from the selection judg- 
ing unit 408., 

[0188] FIG 27 is a block diagram showing details of 
the character direction detecting unit 403 shown in FIG. 
23, FIG. 28 is a flowchart showing processes executed 20 
in the character direction detecting unit 403. Hereinafter, 
explanations will be given of the character direction de- 
tecting unit 403 with reference to FIGs. 27 and 28, 
[01 89] The character image outputted from the docu- 
ment direction selecting unit 402 is inputted into the 25 
character image storing unit 501 and temporarily stored 
therein (S201 ). The character image converting unit 502 
receives the character image inputted into the character 
image storing unit 501 , and applies the rotation process- 
es of 90°, 180° and 270° and the mirroring process to 30 
the inputted character image (S202) The character im- 
age of which the character direction is converted is sent 
to the character image storing unit 501 again and stored 
therein (S203). 

[0190] The character image inputted into the charac- 3S 
ter image storing unit 501 is successively inputted into 
the character recognizing unit 503. The character rec- 
ognizing unit 503 carries out a well-known character rec- 
ognition process (S204), and outputs a candidate char- 
acter and a reliability obtained as the result of the char- 40 
acter recognition for each direction to the character di- 
rection determining part 504 (S205). 
[0191] Now, explanations will be given of timing be- 
tween the character recognition process in the character 
recognizing unit 503 and the conversion process of the 45 
character image direction in the character image con- 
verting unit 502, 

[0192] When the character image inputted from the 
character image selecting unit 401 is stored in the char- 
acter image storing unit 501 , the character recognizing so 
unit 503 recognizes the inputted character image before 
the character image converting unit 502 changes the di- 
rection of the character image When the first character 
recognition is finished, the character image converting 
unit 502 execute the first character direction conversion ss 
(for example, rotation of 90°), and the character recog- 
nizing unit 503 executes the second character recogni- 
tion for the character to which the character direction 



conversion is applied Successively, the character im- 
age converter 502 executes the second character direc- 
tion conversion (rotation of 180°), and the character rec- 
ognizing unit 503 executes the third character recogni- 
tion. 

[01 93] In this way the character recognition is execut- 
ed whenever the character direction conversion is exe- 
cuted, 

[01 94] Output from the character recognizing unit 503 
is inputted into the character direction determining unit 
504, The character direction determining unit 504 se- 
quentially instructs the character image converting unit 
502 to convert the character image direction whenever 
receiving the output from the character recognizing unit 
503. Further, the character direction determining unit 

504 receives the candidate character and the reliability 
from the character recognizing unit 503, and judges 
whether these satisfy the predetermined condition 
(S206). When the candidate characterand the reliability 
satisfy the predetermined condition, the character direc- 
tion determining unit 504 determines the direction of the 
character image based on, in addition to the candidate 
character and the reliability, the character direction con- 
verting method in which the candidate characterand the 
reliability are obtained and outputs the direction of the 
character image (S207)„ 

[0195] Additionally, when no reliability or nothing sat- 
isfies the predetermined condition though the character 
direction is converted into any direction, the character 
direction determining unit 504 instructs the character im- 
age extracting unit 401 to clip the next character image 
(S208). 

[01 96] FIG.. 29 is block diagram showing more details 
of the character direction determining unit 404 shown in 
FiG 23. FIG. 30 is a flowchart showing processes exe- 
cuted in the character direction determining unit 404. 
Hereinafter, explanations will be given of the action of 
the character direction determining unit 404 with refer- 
ence to FIGs., 29 and 30, 

[0197] The reliability outputted from the character di- 
rection detecting unit 403 is inputted into the maximum 
reliability detecting unit 505 and the reliability comparing 
unit 506 in parallel The candidate character outputted 
from the character direction detecting unit 403 is input- 
ted into the character comparing unit 508., Further, in- 
formation concerning the direction conversion outputted 
from the character direction detecting unit 403 is input- 
ted into the detection result judging unit 509. 
[0198] First, the maximum reliability deiecting unit 

505 receives reliabilities of respective directions about 
specified character images to be inputted„ Then, the 
maximum reliability detecting unit 505 detects the max- 
imum reliability among these reliabilities and outputs it 
to the detection result judging unit 509 (S301 ). Further, 
a threshold is previously set in the reliability comparing 
unit 506, and the reliability comparing unit 506 com- 
pares the inputted reliability with this threshold sequen- 
tially (S302) When there is a reliability exceeding the 
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threshold, the number of reliabilities is counted (S303)., 
About one character image, when the number of relia- 
bilities exceeding the threshold is over a predetermined 
number such as two (S304), the reliability comparing 
unit 506 notifies the detection result judging unit 509 of 
an error (S305). 

[01 99] The detection result judging unit 209 receives 
the maximum reliability from the maximum reliability de- 
tecting unit 505 and the error notice from the reliability 
comparing unit 506, The detection result judging unit 
509, when receiving the error notice from the reliability 
comparing unit 506, terminates the direction determin- 
ing process as to the character image in spite of the mid- 
dle of the character direction determining process, and 
instructs the character image extracting unit 401 to clip 
the next character image (S306) 
[0200] On the other hand, the detection result judging 
unit 509, when receiving no error notice, instructs the 
character direction detecting unit 403 tp /convert the 
character image. 

[0201] After instructing to convert one character as to 
all directions, the detection result judging unit 509 com- 
pares the maximum reliability detected by the maximum 
reliability detecting unit 505 with the predetermined 
threshold (S307) When the maximum reliability is less 
than the.predetermined threshold, no direction determi- 
nation is executed about the character image., Then, the 
detection result judging unit 509 instructs the character 
image extracting unit 401 to clip the next character im- 
age (S306). 

[0202] When the maximum reliability exceeds the 
threshold, the detection result judging unit 509 notifies 
the character comparing unit 508 of the character image 
direction at which the maximum reliability is obtained 
(S308). The character comparing unit 508 selects a can- 
didate character of the specified direction, compares the 
candidate character with the character stored in the 
character table 507 (S309), and judges whether or not 
a coincident character exists (S310). 
[0203] The character table 507 stores a list of charac- 
ters to which no character direction detection is applied 
because of symmetry as to right and left or the like., The 
character comparing unit 508 compares the candidate 
character with the characters stored in the character ta- 
ble 507, thereby judging whether the candidate charac- 
ter can be used to detect the character direction or not, 
[0204] When the coincident character exists in the 
character table 507, the character comparing unit 508 
notifies the detection result judging unit 509 of that ef- 
fect. In response to this, the detection result judging unit 
509 judges that no direction determination is executed 
as to this -character image, and instructs the character 
image extracting unit 401 to clip the next character im- 
age. 

[0205] On the other hand, there is no character coin- 
cident with the candidate character in the character ta- 
ble 507, the character comparing unit 508 notifies the 
detection result judging unit 509 of that effect In re- 



sponse to this, the detection result judging unit 509 out- 
puts the character direction of this character and the 
maximum reliability (S311). 

[0206] The outputted character direction and maxi- 
s mum reliability are used to erect the document image 
[0207] As above described, according to the second 
embodiment, a character having a high recognition rate 
in the character recognition (such as a Chinese charac- 
ter) is selectively extracted to judge the character direc- 
10 tion, and the character direction is judged based on the 
extracted character Thus, it is possible to improve the 
accuracy of the character direction judgement rather 
than the conventional technique 

[0208] This invention being thus described, it will be 
15 obvious that same may be varied in various ways. Such 
variations are not to be regarded as departure from the 
spirit and scope of the invention, and ail such modifica- 
tions would be obvious for one skilled in the art intended 
to be included within the scope of the following claims.. 
20 [0209] It will be appreciated that computer programs 
can be stored on a variety of media, such as floppy 
disks, hard disks, CD-ROMs, etc, or downloaded into 
a computer, for example from the Internet. It is intended 
that the attached claims should encompass a computer 
25 program by itself or in any other form. 



Claims 

30 1. A image status detecting apparatus detecting 
whether or not a character image inputted through 
image input means is mirrored horizontally or verti- 
cally and/or what degree the character image is ro- 
tated compared with an original character, said im- 

3S age status detecting apparatus comprising; 

image converting means for rotating and for 
mirroring the character image inputted through 
the image input means, 

40 character recognizing means for referring a 

character recognition dictionary storing charac- 
ter feature information and for obtaining candi- 
date characters resemble to the character im- 
age mirrored and rotated by the image convert- 

45 ing means and a recognition rate indicating a 

degree of recognition certainty; and 
conversion procedure detecting means for de- 
tecting, based on the candidate characters and 
the recognition rate obtained by the character 

50 recognizing means, a rotation angle and/or an 

existence of mirroring needed to direct the 
character image to a proper direction. 

2. A document image correcting apparatus compris- 
es ing: 

character image storing means for storing a 
character image inputted through image input 
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means; 

image status detecting means for detecting 
whether or not the character image stored in 
the character image storing means is mirrored 
horizontally or vertically and what degree the s 
character image is rotated compared with an 
original character; and 

image correcting means for directing the char- 
acter image stored in the character image stor- 
ing means to a proper direction based on a de- 10 
tected result of the image status detecting 
means. 

A document image correcting apparatus according 

to Claim 2, said image status detecting means com- is 

prising; 

image converting means for rotating and for 
mirroring the character image inputted through 
the image input means; 20 
character recognizing means for referring a 
character recognition dictionary storing charac- 
ter feature information and for obtaining candi- 
date characters resemble to the character im- 
age mirrored and rotated by the image convert- 2S 
ing means and a recognition rate indicating a 
degree of recognition certainty; and 
conversion procedure detecting means for de- 
tecting, based on the candidate characters and 
the recognition rate obtained by the character 30 
recognizing means, a rotation angle and an ex- 
istence of mirroring needed to direct the original 
character image to the proper direction 

A document image correcting apparatus according 35 
to Claim 2 or 3, further comprising 

document image storing means for storing a 
document image including a character line 
which consists of plural characters inputted 40 
through the image input means; and 
character image extracting means for extract- 
ing at least one character image from the doc- 
ument image inputted through the input image 
means; 45 
wherein said character image storing means 
stores the character extracted by the character 
image extracting means, and wherein said im- 
age correcting means corrects the document 
image stored in said document image storing so 
means to a proper image in a specified direction 
in accordance with a detected result of the im- 
age status detecting means., 

A document image correcting apparatus according 55 
to any one of Claims 2 to 4, further comprising: 

conversion controlling means for controlling 
said image converting means 



6. A document image correcting apparatus according 
to Claim 3, wherein said conversion procedure de- 
tecting means detects a rotated angle and/or an ex- 
istence of mirroring in a case that the recognition 
rate of the character image obtained by the charac- 
ter recognizing means becomes maximum among 
all combinations of rotation and mirroring executed 
by the image converting means, 

7. A document image correcting apparatus according 
to Claim 3, wherein said conversion procedure de- 
tecting means detects a rotated angle and/or an ex- 
istence of mirroring when the recognition rate ob- 
tained by the character recognizing means is over 
a predetermined value 

8. A document image correcting apparatus according 
to Claim 5, wherein said conversion controlling 
means controls the image converting means so that 
the character image is converted in a state that ro- 
tation and mirroring are combined in a specified or- 
der. 

9. A document image correcting apparatus according 
to Claim 5, wherein said conversion controlling 
means controls the image converting means so that 
the character image is converted in accordance 
with only a specified combination of rotation and 
mirroring among plural combinations of rotated an- 
gles and mirroring 

10. A document image correcting apparatus according 
to Claim 5, wherein said conversion controlling 
means controls the image converting means so that 
the character image is converted in order of high 
frequency among combinations of rotated angles 
and mirroring which previously make character im- 
ages erecting. 

11. A document image correcting apparatus according 
to Claim 5, wherein said conversion controlling 
means controls the image converting means so that 
a character image is rotated every predetermined 
angle while mirrored. 

12. A document image correcting apparatus according 
to Claim 5, wherein said image correcting means 
does not correct character images not satisfying a 
specified condition as a result of character recogni- 
tion as to all character images converted by said 
image converting means, 

13. An image status detecting method comprising: 

an image converting step of rotating and/or of 
mirroring horizontally or vertically the character 
image inputted through the image input tool; 
a character recognizing step of referring a char- 
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acter recognition dictionary storing character 
feature information and of obtaining candidate 
characters resembie to the character image 
mirrored and rotated in the image converting 
step and a recognition rate indicating a degree 
of recognition certainty; and 
a conversion procedure detecting step of de- 
tecting, based on the candidate characters and 
the recognition rate obtained in the character 
recognizing step, a rotation angle and an exist- 
ence of mirroring needed to direct the original 
character image to a proper direction. 

14. A document image correcting method comprising: 

a character image storing step of storing a char- 
acter image inputted through an image input 
tool, 

an image status detecting step of detecting 
whether or not the character image stored in 
the character image storing step is mirrored 
horizontally or vertically and/or what degree the 
character image is rotated compared with an 
original character; and 

an image correcting step of directing the char- 
acter image stored in the character image stor- 
ing step to a proper direction based on a de- 
tected result of the image status detecting step, 

15. A computer readable medium storing a program, 
said program comprising: 

an image converting step of rotating and/or of 
mirroring horizontally or vertically the character 
image inputted through the image input tool; 
a character recognizing step of referring a char- 
acter recognition dictionary storing character 
feature information and of obtaining candidate 
characters resemble to the character image 
mirrored and rotated in the image converting 
step and a recognition rate indicating a degree 
of recognition certainty; and 
a conversion procedure detecting step of de- 
tecting, based on the candidate characters and 
the recognition rate obtained in the character 
recognizing step, a rotation angle and an exist- 
ence of mirroring needed to direct the original 
character image to a proper direction. 

16. A computer readable medium storing a program, 
said program comprising: 

a character image storing step of storing a char- 
acter image inputted through an image input 
tool; 

an image status detecting step of detecting 
whether or not the character image stored in 
the character image storing step is mirrored 



horizontally or vertically and/or what degree the 
character image is rotated compared with an 
original character; and 

an image correcting step of directing the char- 
s acter image stored in the character image stor- 

ing step to a proper direction based on a de- 
tected result of the image status detecting step, 

17. A document image correcting apparatus compris- 
10 ing: 

character image extracting means for extract- 
ing a character image satisfying a predeter- 
mined condition from an inputted document im- 
15 age; and 

character direction detecting means for detect- 
ing a rotation and an existence of mirroring as 
to the character image extracted by the char- 
acter image extracting means. 

20 

18. A document image correcting apparatus according 
to Claim 17, wherein the predetermined condition 
is a ratio of a number of black pixels to a number of 
pixels in the character image extracted by the char- 
ts acter image extracting means., 

19. A document image correcting apparatus according 
to Claim 17, wherein the predetermined condition 
is a length ratio of a long side to a short side of the 

30 character image extracted by the character image 
extracting means, 

20. A document image correcting apparatus according 
to Claim 17, wherein the predetermined condition 

35 is at least one of a number of pixels in a lengthwise 
direction and a number of pixels in a horizontal di- 
rection in the character image extracted by the char- 
acter image extracting means 

40 21. A document image correcting apparatus according 
to any one of Claims 1 7 to 20, wherein said charac- 
ter image extracting means, when a character im- 
age having a high symmetry appears, does not ex- 
tract the character image. 

45 

22. A document image correcting apparatus according 
to any one of Claims 1 7 to 21 , wherein said charac- 
ter direction detecting means applies a character 
recognition to the character image extracted by the 

50 character image extracting means, and when a can- 
didate character obtained as a result of the charac- 
ter recognition coincides with a predetermined 
character, does not detect a direction of the char- 
acter image, 

55 

23. A document image correcting apparatus compris- 
ing 
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character image extracting means for selec- 
tively extracting a character image satisfying a 
predetermined condition from an inputted doc- 
ument image; 

character image converting means for applying 
a rotation process at a predetermined angle to 
the character image extracted by said charac- 
ter image extracting means, for applying the ro- 
tation process to the character image after ap- 
plying a mirroring process, and for outputting 
the character image to which the rotation proc- 
ess and the mirroring process are applied; 
character recognizing means for executing a 
character recognition as to each direction of the 
character image outputted from said character 
image converting means; and 
document direction determining means for de- 
termining a direction of the inputted document 
image based on a result of the character rec- 
ognition executed by said character recogniz- 
ing means. 

24. A document image correcting apparatus according 
to Claim 23, wherein said character recognizing 
means outputs reiiability information indicating an 
accuracy of the character recognition process as to 
each direction of the character image outputted 
from the character image converting means, and 
wherein said document direction determining 
means determines the direction of the inputted doc- 
ument image based on the reliability information of 
the character image outputted from said character 
recognizing means. 

25. A document image correcting apparatus according 
to Claim 23 or 24, wherein said document direction 
determining means detects a maximum reiiability 
information among the reliability information input- 
ted from said character recognizing means and de- 
termines the direction of the inputted document im- 
age based on the maximum reliability information 

26. A document image correcting apparatus according 
to Claim 23 or 24, wherein said document direction 
determining means compares the reiiability infor- 
mation inputted from said character recognizing 
means with a predetermined threshold, and exe- 
cutes no document direction determination based 
on the character image when no reliability informa- 
tion exceeds the predetermined threshold as to 
each direction of the character image. 

27. A document image correcting apparatus according 
to Ciaim 23 or 24, wherein said document direction 
determining means compares the reiiability infor- 
mation inputted from said character recognizing 
means with a predetermined threshold, and exe- 
cutes no document direction determination based 



on the character image when a number of reliability 
information exceeds a predetermined number 

28. A document image correcting apparatus according 
s to Claim 23 or 24, wherein said document direction 

determining means detects directions of characters 
as to plural character images and determines the 
direction of the document image based on the di- 
rections which are detected. 

10 

29. A document image correcting apparatus according 
to Ciaim 28, wherein said document direction deter- 
mining means calculates an average of reliabilities 
of each character in each character direction as to 

is plural character images, and determines the direc- 
tion of the document image based on the average. 

30. A document image correcting apparatus according 
to Claim 28, wherein said document direction deter- 
ge mining means determines a character direction ob- 
taining a most number of maximum reliabilities 
among character directions having plural maximum 
reliabilities of each character image as a document 
direction. 

25 

31 . A document image correcting apparatus according 
to Claim 28, wherein said document direction deter- 
mining means, when maximum reliabilities are ob- 
tained more than a predetermined number as to a 

30 character direction, determines the character direc- 
tion as a document image direction. 

32. A document image correcting method comprising: 

35 a step of extracting a character image satisfying 

a predetermined condition from an inputted 
document image; and 

a step of judging rotation and/or an existence 
of mirroring as to the character image which is 
40 selectively extracted.. 

33. A document image correcting method comprising: 

a step of selectively extracting a character im- 
45 age satisfying a predetermined condition from 

an inputted document image; 
a step of applying a rotation process at a pre- 
determined angle to the character image which 
is selectively extracted and of applying the ro- 
se tation process to the character image after ap- 
plying a mirroring process: 
a step of outputting the character image to 
which the rotation process and the mirroring 
process are applied; 
55 a step of executing a character recognition as 
to each direction of the character image which 
is outputted; and 

a step of determining a direction of the inputted 
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document image based on a result of the char- 
acter recognition, 

34. An information storage medium storing a program, 
said program comprising; 

a step of extracting a character image satisfying 
a predetermined condition from an inputted 
document image; and 

a step of judging a rotation and an existence of 
mirroring as to the character image which is se- 
lectively extracted 

35. An information storage medium storing a program; 
said program comprising: 

a step of selectively extracting a character im- 
age satisfying a predetermined condition from 
an inputted document image; 
a step of applying a rotation process at a pre- 
determined angle to the character image which 
is selectively extracted and of applying the ro- 
tation process to the character image after ap- 
plying a mirroring process; 
a step of outputting the character image to 
which the rotation process and the mirroring 
process are applied; 

a step of executing a character recognition as 
to each direction of the character image which 
is outputted; and 

a step of determining a direction of the inputted 
document image based on a result of the char- 
acter recognition, 

36. A status detecting apparatus detecting what status 
a document image inputted through image input 
means is as to a proper direction, said status de- 
tecting apparatus comprising: 

image converting means for producing a rotat- 
ed image or a mirrored image from the docu- 
ment image inputted through the image input 
means; 

character recognizing means for applying a 
character recognition process to each charac- 
ter included in the rotated image and the mir- 
rored image produced by the image converting 
means, and 

conversion procedure detecting means for ar- 
ranging the document image inputted through 
the image input means in the proper direction 
based on an image direction of which a char- 
acter recognition rate is highest among the ro- 
tated image and the mirrored image as a result 
of character recognition by the character rec- 
ognizing means. 

37. A document image correcting apparatus compris- 



input means for inputting a document image; 
selecting means for selecting a character im- 
s age from the document image inputted through 

the input means; 

judging means for judging which direction the 
selected character image is inputted as to a 
standard direction; and 
10 arranging means for arranging all of the input- 

ted document image into the standard direction 
based on the judged direction, 

38. A document image correcting apparatus compris- 
es ing: 

input means for inputting:a document image; 
judging means for judging the inputted docu- 
ment image and an existence of mirroring as to 
20 a standard direction, and 

applying means for applying a process includ- 
ing mirroring to the document image based on 
a direction of the document image which is 
judged. 

2S 

39. A document image correcting. method comprising: 

a step of inputting a character; 

a step of rotating the inputted character in a plu- 
30 rality directions and of mirroring the character 

rotated in each direction; 

a step of executing character recognition as to 

the rotated and mirrored character; 

a step of selecting a character of which a rec- 
35 ognition rate of the character recognition is 

highest; and 

a step of converting the inputted character into 
a proper direction based on a rotation direction 
and an existence of mirroring of the selected 
40 character 

40. A computer program which, when run on a compu- 
ter, causes the computer to carry out a method as 
claimed in any one of claims 13, 14, 32, 33 and 39. 



45 

41. A computer program which, when loaded into a 
computer, causes the computer to become appara- 
tus as claimed in any one of claims 1 to 12, 17 to 
31 and 36 to 38. 

so 
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